Risk-Averse Approximate Dynamic Programming with Quantile-Based Risk Measures
نویسندگان
چکیده
In this paper, we consider a finite-horizon Markov decision process (MDP) for which the objective at each stage is to minimize a quantile-based risk measure (QBRM) of the sequence of future costs; we call the overall objective a dynamic quantile-based risk measure (DQBRM). In particular, we consider optimizing dynamic risk measures where the one-step risk measures are QBRMs, a class of risk measures that includes the popular value at risk (VaR) and the conditional value at risk (CVaR). Although there is considerable theoretical development of risk-averse MDPs in the literature, the computational challenges have not been explored as thoroughly. We propose a datadriven or simulation-based approximate dynamic programming (ADP) algorithm to solve the risk-averse sequential decision problem. In addition, we address the issue of inefficient sampling for risk applications in simulated settings and present a procedure, based on importance sampling, to direct samples toward the “risky region” as the ADP algorithm progresses. Finally, we show numerical results of our algorithms in the context of an application involving risk-averse bidding for energy storage.
منابع مشابه
SDDP for multistage stochastic linear programs based on spectral risk measures
We consider risk-averse formulations of multistage stochastic linear programs. For these formulations, based on convex combinations of spectral risk measures, risk-averse dynamic programming equations can be written. As a result, the Stochastic Dual Dynamic Programming (SDDP) algorithm can be used to obtain approximations of the corresponding risk-averse recourse functions. This allows us to de...
متن کاملSampling-Based Decomposition Methods for Multistage Stochastic Programs Based on Extended Polyhedral Risk Measures
We define a risk-averse nonanticipative feasible policy for multistage stochastic programs and propose a methodology to implement it. The approach is based on dynamic programming equations written for a risk-averse formulation of the problem. This formulation relies on a new class of multiperiod risk functionals called extended polyhedral risk measures. Dual representations of such risk functio...
متن کاملA Risk-averse Inventory-based Supply Chain Protection Problem with Adapted Stochastic Measures under Intentional Facility Disruptions: Decomposition and Hybrid Algorithms
Owing to rising intentional events, supply chain disruptions have been considered by setting up a game between two players, namely, a designer and an interdictor contesting on minimizing and maximizing total cost, respectively. The previous studies have found the equilibrium solution by taking transportation, penalty and restoration cost into account. To contribute further, we examine how incor...
متن کاملRisk-Averse Control of Undiscounted Transient Markov Models
We use Markov risk measures to formulate a risk-averse version of the undiscounted total cost problem for a transient controlled Markov process. We derive risk-averse dynamic programming equations and we show that a randomized policy may be strictly better than deterministic policies, when risk measures are employed. We illustrate the results on an optimal stopping problem and an organ transpla...
متن کاملFirm Specific Risk and Return: Quantile Regression Application
The present study aims at investigating the relationship between firm specific risk and stock return using cross-sectional quantile regression. In order to study the power of firm specific risk in explaining cross-sectional return, a combination of Fama-Macbeth (1973) model and quantile regression is used. To this aim, a sample of 270 firms listed in Tehran Stock Exchange during 1999-2010 was i...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2015